Visual Saliency Weighting and Cross-Domain Manifold Ranking for Sketch-Based Image Retrieval

A Sketch-Based Image Retrieval (SBIR) algorithm compares a linedrawing sketch with images. The comparison is made difficult by image background clutter. A query sketch includes an object of interest only, while database images would also contain background clutters. In addition, variability of hand-drawn sketches, due to “stroke noise” such as disconnected and/or wobbly lines, also makes the comparison difficult. Our proposed SBIR algorithm compares edges detected in an image with lines in a sketch. To emphasize presumed object of interest and disregard backgrounds, we employ Visual Saliency Weighting (VSW) of edges in the database image. To effectively compare the sketch containing stroke noise with database images, we employ Cross-Domain Manifold Ranking (CDMR), a manifold-based distance metric learning algorithm. Our experimental evaluation using two SBIR benchmarks showed that the combination of VSW and CDMR significantly improves retrieval accuracy.